Snapshot Testing the Accessibility Tree

We are going to write a test that creates a snapshot of the Accessibility Tree and isolates the header element.

Since screen reader output is not something we can assert programmatically quite yet, snapshot testing techniques for the Accessibility Tree can help us approximate it.

Also, limiting the scope of the test to a smaller part of the Accessibility Tree will help make it less brittle. If something changes elsewhere on a page, it will be less likely that you’ll have to rewrite all your snapshot tests.

There is a script already set up in package.json that will start Puppeteer via jest-puppeteer.

yarn test:puppeteer

yarn test:puppeteer

Adding the --watch flag will re-run tests as the file is saved.

There is also a jest-puppeteer.config.js file that is used to start up the local dev server and launch Puppeteer’s Chromium browser instance with support for experimental web platform features, but without the sandbox.

module.exports = {
    server: {
      command: 'parcel index.html'
    },
    launch: {
      args: ['--no-sandbox', '--enable-experimental-web-platform-features'],
    }
}

module.exports = {
    server: {
      command: 'parcel index.html'
    },
    launch: {
      args: ['--no-sandbox', '--enable-experimental-web-platform-features'],
    }
}

Writing the First Test

Inside of the exercise4-puppeteer directory is

The first thing to do at the top of the file is tell Jest we want to use the Puppeteer environment as opposed to JSDOM.

We’ll import expect-puppeteer as well as path and fs from the Node.js filesystem APIs:

/**
/* @jest-environment puppeteer
**/

import 'expect-puppeteer'
import path from 'path'
import fs from 'fs'

/**
/* @jest-environment puppeteer
**/

import 'expect-puppeteer'
import path from 'path'
import fs from 'fs'

As we’ve seen in other test files we’ve written, we’ll write a describe statement and a beforeAll block inside to tell Puppeteer to navigate to the Passes page on the local dev server.

This time it’s a little different as the beforeAll callback function will be asynchronous using async/await. Put the async keyword at the start of the callback function to invoke this behavior. Inside of the function body, we’ll await a call to page.goto the local development server.

The first test we’ll write will be an it block that says that the “Pick a Plan and Start Your Adventure Today!” heading text should appear somewhere on the page. This will also have an async function as the callback.

Referencing the expect-puppeteer docs, we can use toMatch to say we expect text to appear on the page.

Here’s what it looks like all together at this point:

import 'expect-puppeteer'
import path from 'path'
import fs from 'fs'

describe('Accessibility Tree', () => {
    beforeAll(async () => {
        await page.goto('http://localhost:1234/passes')
    })

    it('should display heading text on page', async () => {
        await expect(page).toMatch('Pick a Plan and Start Your Adventure Today!')
    })
})

import 'expect-puppeteer'
import path from 'path'
import fs from 'fs'

describe('Accessibility Tree', () => {
    beforeAll(async () => {
        await page.goto('http://localhost:1234/passes')
    })

    it('should display heading text on page', async () => {
        await expect(page).toMatch('Pick a Plan and Start Your Adventure Today!')
    })
})

Running the Puppeteer tests with yarn test:puppeteer, we can see this test passes!

Writing the Tree Snapshot Test

Now let’s write a test that writes a snapshot of the Accessibility Tree.

To create the snapshot, we’ll call page.accessibility.snapshot(). There are some options that can be passed in. We’ll use them later.

To save the file, first create a variable for the file path by calling path.join(process.cwd()) to get the current working directory. Further arguments to path.join will create subdirectories and the last arg will be the filename to write.

From there, we’ll use fs.writeFile and pass in the file path to the exercise directory and the stringified JSON output of the snapshot.

it('should create a JSON file for the accessibility tree', async() => {
        const snapshot = await page.accessibility.snapshot()

        const assetFilePath = path.join(process.cwd(), 'exercise4-puppeteer', 'a11y-tree.json')
        fs.writeFile(
            assetFilePath,
            JSON.stringify(snapshot, null, 5), 
            () => {}
        )
 })

it('should create a JSON file for the accessibility tree', async() => {
        const snapshot = await page.accessibility.snapshot()

        const assetFilePath = path.join(process.cwd(), 'exercise4-puppeteer', 'a11y-tree.json')
        fs.writeFile(
            assetFilePath,
            JSON.stringify(snapshot, null, 5), 
            () => {}
        )
 })

Because this is a test file, we should include an actual test.

We will write an assertion that the file was created, even though we can see it show up in the sidebar when we run the test command.

Using the fs.stat() Node filesystem API, we can pass in the file path we created and inside of the callback add an assertion that the file was successfully created:

fs.stat(assetFilePath, (err, stat) => {
	expect(stat.isFile()).toBe(true)
})

fs.stat(assetFilePath, (err, stat) => {
	expect(stat.isFile()).toBe(true)
})

Once we run the test command, the a11y-tree.json file should be created.

Writing a Snapshot Comparison Test

Inside of a11y-tree.test.js, add another it block where we will test if the current page snapshot matches a previous snapshot.

We’ll create a variable for the called axTree that will take a snapshot of the current page. Then we’ll add an expect that checks it against a saved a11y-tree.test.js.snap file inside the __snapshots__ directory.

it(`has the correct axTree`, async () => {
	const axTree = await page.accessibility.snapshot()
	expect(axTree).toMatchSnapshot()
})

it(`has the correct axTree`, async () => {
	const axTree = await page.accessibility.snapshot()
	expect(axTree).toMatchSnapshot()
})

When the test runs it fails, as the older snapshot is out of date.

But we don’t want to fix this test by undoing all of our work on the site’s markup!

Instead, at the bottom of the error message Jest instructs us to add a -u flag to the test command to update the saved snapshot:

Running the command and updating the snapshot results in a passing test:

Making Snapshots More Interesting

The a11y-tree.json file resulting from running the test looks like this:

{
     "role": "RootWebArea",
     "name": "CampSpots Passes",
     "children": [
          {
               "role": "heading",
               "name": "CampSpots",
               "level": 1
          },
          {
               "role": "link",
               "name": "CampSpots"
          },
          {
               "role": "heading",
               "name": "Plan Your Trip",
               "level": 2
          },
					...

{
     "role": "RootWebArea",
     "name": "CampSpots Passes",
     "children": [
          {
               "role": "heading",
               "name": "CampSpots",
               "level": 1
          },
          {
               "role": "link",
               "name": "CampSpots"
          },
          {
               "role": "heading",
               "name": "Plan Your Trip",
               "level": 2
          },
					...

To get more information, we can pass a config object into page.accessibility.snapshot() that sets the option to be false:

const snapshot = await page.accessibility.snapshot({
	interestingOnly: false
})

const snapshot = await page.accessibility.snapshot({
	interestingOnly: false
})

Now when re-running the test, the JSON output includes a lot more information than it did before:

{
  "role": "RootWebArea",
  "name": "CampSpots Passes",
  "children": [
    {
      "role": "generic",
      "name": "",
      "children": [
        {
          "role": "generic",
          "name": "",
          "children": [
            {
              "role": "generic",
              "name": "",
              "children": [
                {
                  "role": "heading",
                  "name": "CampSpots",
                  "level": 1,
                  "children": [
                    {
                      "role": "StaticText",
                      "name": "CampSpots"
                    }
                  ]
                }
              ]

{
  "role": "RootWebArea",
  "name": "CampSpots Passes",
  "children": [
    {
      "role": "generic",
      "name": "",
      "children": [
        {
          "role": "generic",
          "name": "",
          "children": [
            {
              "role": "generic",
              "name": "",
              "children": [
                {
                  "role": "heading",
                  "name": "CampSpots",
                  "level": 1,
                  "children": [
                    {
                      "role": "StaticText",
                      "name": "CampSpots"
                    }
                  ]
                }
              ]

Almost by definition, “generic” is not interesting, but it is part of the structure. It’s useful to look at the “before and after” of using the interestingOnly option. Depending on the test you’re writing, it might be a better idea to only capture the more pertinent information.

Final Thoughts on jest-puppeteer

In most cases, I would suggest sticking with Cypress and Jest for the types of tests we’ve been working with so far. However, there might be a situation in your work where you would benefit from being able to test against an entire page snapshot.